This invention generally concerns unmanned aerial vehicles (“UAVs”), such as planes, helicopters, balloons, and the like, that are operated without need of a person being onboard.
Current large-scale UAVs, defined to be those with wingspans greater than twenty feet (e.g., the Predator and the Global Hawk) have been developed to fulfill tactical combat support roles, such as high-altitude surveillance and munitions delivery, while keeping the human pilot out of harm's way. These planes were designed to be similar in function to their on-board, human-piloted counterparts, typically having payloads of several hundred pounds, running on fuel-powered engines, and being capable of flying at high altitudes. With their large wingspan and powerful engines, these aircraft can carry the full sensor suite of a tactical aircraft and can have the additional capability to carry radars, cameras for surveillance, other environmental sensors, and even missiles and bombs, without affecting flight performance. These planes possess a small degree of autonomy (e.g., waypoint navigation, auto-takeoff, and auto-landing) but require a human in direct control from a remote station for the majority of their operations.
Mid-sized UAVs, those with wingspans from twelve to twenty feet, such as the Pointer and the Shadow, have been developed not to replace the tactical combat aircraft in service today, but to supplement the soldiers on the ground, and, in some cases, even to replace human scouts. More specifically, they were developed to provide a highly-specialized observation tool to be deployed and flown by a small group of soldiers for the purpose of inspecting forward areas or for early detection of approaching threats. These aircraft are essentially electrically-powered, remote control airplanes that may be outfitted electric motors, they are well-suited for discrete, short-range surveillance. They are portable through disassembly and the components may be carried in multiple backpacks. With a few minutes of setup time, they can be launched and provide an aerial view of the surrounding area. Some mid-sized UAVs are similar to their larger counterparts in that they have limited autonomy and are capable of performing waypoint navigation and altitude hold, which can assist their remotely located human pilots in gathering aerial video.
Small-sized UAVs, those with wingspans of less than twelve feet, have recently evolved from mid-sized UAVs. The main goal in developing small-sized UAVs has been to create systems that can be carried, launched, and operated by a single person. These systems use detachable or foldable wings, can be stored in a single backpack, rather than several, and can be launched by hand. These small aircraft are also well suited for covert operations, because they are quiet, small enough to fly into open windows or small caves, and appear as birds from a distance.
Current UAVs are limited in that, while capable of basic semi-autonomous operation, they have not yet been able to make the transition to a fully autonomous state. The principal source of this limitation is the fact that, while they can generally maneuver in an open-air environment by remote control or simple waypoint navigation, they cannot fly in the presence of obstacles such as buildings, trees, or towers, because their navigation systems are based on position alone. Primarily, this position is derived from a GPS and corresponds to a global coordinate system. These known global coordinates are used as waypoint markers by which a UAV finds its way to a destination.
Problems arise when obstacles are present between waypoint markers. One solution to this problem involves associating GPS coordinates with geographical relief maps in order to avoid areas where known geographic features are the source of obstacles. This technique offers a solution by confining the flight paths to open-air environments. It does not, however, free the aircraft to safely explore areas where the terrain is dynamic, where obstacles are largely unmapped, or where the environment is physically cluttered (the streets of a city or town, for example). In order for a UAV to enter and successfully navigate such environments, a more advanced system is needed.
Some much larger aircraft use radar-type systems to provide high-resolution, high-accuracy three-dimensional images to aid in navigation. However, radar-type systems require payload and power capabilities orders of magnitude larger than what UAVs can support. These systems also emit large levels of detectable energy during their operation.
Recent work in UAV computer vision can be divided into two major focus areas: two-dimensional analysis and three-dimensional analysis. Two-dimensional analysis usually refers to color-based, texture-based, or simple structural-based processing methods. These methods allow for image segmentation (i.e., isolating the interesting portion to reduce bandwidth) and feature extraction (i.e., identifying structures like points, lines and simple objects).
When applied in the realm of UAVs, extracting features such as the straight line of the horizon yields a vision-based approach to stabilization, and extracting features such as points or lines in the middle of an image coming from a forward-looking camera can yield a primitive obstacle detection system, such as that described in U.S. Pat. No. 5,581,250. This patent, which notes that the system disclosed therein can be used in conjunction with a UAV that utilizes an inertial reference system, also explains that the collision avoidance capability can be disabled, meaning that the obstacle detection system of the invention is a system independent from any external devices. This system provides no means to navigate in an unknown or dynamic environment or to adapt to the presence of multiple obstacles.
Three-dimensional image analysis builds on two-dimensional analysis concepts by correlating good features (e.g., image points) across multiple images to recover the three-dimensional motion of the camera and the structure of the observed scene. The recovered structure of the scene can then be used to plan the path of flight. These multiple images can be acquired simultaneously (i.e., stereo cameras) or sequentially (i.e., single camera moving relative to the scene). The reconstructions (or estimations) obtained from the feature correspondences between frames are subject to a high-degree of projective ambiguity. When compared with a native three-dimensional radar-type imaging system output, the three-dimension computer vision scene estimates, and camera motions, are low-resolution and very prone to instantaneous noise (large variations between frames).
An application of a three-dimensional image analysis to obstacle detection is described in U.S. Pat. No. 6,678,394. In this system, object distances are determined from corresponding features in simultaneously acquired (stereo) images. To mitigate resolution and noise issues, a three-dimensional reference model is created off-line in a training phase, and used as a comparison threshold for estimated object distance during run-time. This system provides a method for determining the presence of an obstacle using a three-dimensional analysis approach, but would fall short in an unknown or dynamic environment, or where multiple obstacles are present, because it relies on a priori knowledge (offline training) that would not be available in those cases.